The present disclosure relates to computer automated handwriting recognition systems and methods.
Unless otherwise indicated herein, the approaches described in this section are not admitted to be prior art by inclusion in this section.
Handwriting recognition has been a difficult problem for a long time, like speech recognition, for similar reasons. A major difficulty of handwriting recognition is the large variation in writing, both for a single writer and from writer to writer. For a single writer, variation is high within a single document and under varying conditions, for example, between a comfortable seated writing position and a standing strained position. Significant variation occurs with informal writing, when hurried, with too much coffee, with shifting position of the hand across the page, with interruptions and other effects. The form of characters depends on characters that precede and follow, called coarticulation.
Between writers there is enormous variation. A major problem is the multi-writer problem. For cell phones, quick training for the owner's individual writing is enough. However, in important applications, e.g., postal address reading, banking or kiosks, and data entry from forms, recognition must function for many people without training.
Humans readers depend on context in important ways, some available for use by machine reading. Some forms of context are currently difficult for machines, like linguistic semantic context. The accuracy of human reading of handwriting depends heavily on the quality of handwriting, and on the words, phrases, and sentences in which the characters are embedded. Human accuracy depends strongly on understanding handwriting, i.e., reading the handwriting. Handwritten characters in nonsense “words” can be illegible for human readers. Experiments show about 4% character error rate for handwritten characters in nonsense words, without lexical context. Some writers were better; some readers were better. Automated recognition of handwriting can also make use of limited understanding, using dictionaries and spell checking. It may not be practical yet to use grammar and semantics in handwriting recognition programs, nor is artificial intelligence sufficiently advanced to use knowledge that a human being might routinely use in deciphering bad handwriting. Writing is often used to communicate to oneself with brief careless notes as reminders. In such cases, the human who wrote the message has a big advantage over computers.
Handwriting recognition software converts character symbols to symbolic character codes (e.g., ASCII (American Standard Code for Information Interchange), Unicode, etc.) that can be manipulated in a word processor, printed, or used in databases and in artificial intelligence programs. Transforming the handwriting signal to text symbols has proven to be difficult.
Various handwriting recognition software programs are available. Some commercially available products have reasonable apparent performance with lexical correction, but they have some problems without lexical correction. In many scenarios, lexicons are inadequate. Some examples are proper nouns, foreign words (writers of non-English languages), words with no unique English transliteration, non-standard words and abbreviations, severe misspellings. Obtaining estimates of the accuracy of these commercial systems is difficult. The software error rate depends heavily on cooperative writers and on the lexicon. An average character error rate of 1% (recognition rate of 99%) is generally considered to be necessary for acceptance by users. In addition, recognition in real time is often desired.
There are a lot of symbols that may need to be recognized. For example, there are more than 100 characters and symbols in English in large block caps, small block caps, printed lower characters, cursive lower characters, and cursive caps. Most of these characters have several or many variants in handwriting, typical or rare. Other languages written with Roman script have diacritics. Machine print has many fonts, corresponding to thousands of visually distinct symbols. Also there are about 100 widely used symbols including punctuation, usual ASCII symbols #@!˜%{circumflex over ( )}&*( )+, currency symbols: $, etc. There are many mathematical symbols. There are many emoticons. All these total a large number, perhaps 5,000 symbols or more for a single script. Devanagari, an Indic script, has about 450 symbols.
The large number of symbols raises problems. Each handwritten symbol has at least one model, and often has several variants. First, the error rate increases with the number of models; thus the accuracy has to increase from that accuracy required to discriminate a small number of characters symbols, say 26, to compensate for the total symbol set. Second, the computational complexity of recognition increases with the number of models.
Typical recognition is based on multi-template matching, i.e., matching fixed models with instances. In this paradigm, recognition is matching against all models. This method has high computational complexity. Methods using neural nets or using parameterized models with estimation of parameters approximate these thousands of variants.
Even with these methods, there will be new, previously unknown variants. They might be from writers from a different country, e.g., English writers from India. Abilities to learn models of new variants is necessary. These capabilities are necessary for learning a script for a multi-script system.
It is desirable that a handwriting recognition system be functional on a smartphone, and have low power consumption. Computational complexity is a challenge for power consumption. Thus, it would be desirable to have a computationally efficient mechanism for computer automated handwriting recognition systems and methods.
Character correction of text is painfully slow, for humans for either handwriting or keyboards. High accuracy is a goal, both for human-interface and low power consumption.